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1, INTRODUCTION 

The fastest way to deliver medications and fluid replacement throughout the body is via the 
intravenous route. Intravenous therapy may be used for fluid replacement (such as correcting dehydration), 
to deliver medications, to correct electrolyte imbalances, and for blood transfusions. In determining the 
location of veins, a nurse usually encounters problems because each person has a different level of difficulty. 
A flexible and real-time system will greatly assist a nurse in accelerating the visualization process. Currently 
there are tools to visualize veins such as AccuVein, VeinViewer, Vein Site but the price is still quite 
expensive [1]. 

Research on vein visualization is generally divided into 2 scopes: for security purposes and medical 
purposes. In medical purposes, research on vein visualization is more aimed at realtime visualization. Juric et 
al develope a vein visualization system using an android tablet as a platform to perform vein image 
processing and visualization in real-time [1]. Similar visualization technique also developed by Al Gozhali et 
al. on the PC platform [2]. The use of additional screen as a visualization media in those researches is 
considered unsuitable because nurses must look at the screen to observe the veins. Therefore, several studies 
have discussed the veins visualization system which is focused on applying visualization with direct 
projections on the patient's skin. Marcotti et al [3] was discussed about developement of veins visualization 
system. In his research, the position of the projector was placed far above the camera to get the correct 
projection. This back-projection method needs a large space and there was a limitation in projection distance 
that allow the user must performing visualization specific distance. Another method to visualize veins image 
was conducted by using a dual camera system to find the depth of a vein [4]. In this research, the distance 
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between objects and acquisition devices has been set at 35 cm. As seen from the projection method, there will 
be a shift in projection if the distance of the object does not match the distance that has been set. With a dual 
camera system, it is possible to know the distance between the device and the object and make adjustments to 
the projection at each distance. However, in his research only focused on a certain distance and the system 
requires very long computation time even though the image processing process is done on computers with 
processors running at 2.6 GHz. Meanwhile, a realtime and stand-alone system for vein visualization 
equipment will make it easier to use. 

From previous studies, it is known that there are still problems related to the computational time and 
mobility of the developed visualization system. Therefore, this study proposes a real-time vein visualization 
system with high mobility in order to make it easier to use by a nurse. All computational processes are 
carried out using a single board computer. For a portable system, the use of a single camera and mini 
projector system was enough to perform the back-projection at a fixed point. However, just one point will be 
difficult to find and will slow down the intravenous process, thus this system must be able to perform back- 
projection correctly at a certain range of distances. Inspired by previous research [5], a concept of the back- 
projection method by adding a distance parameter to adjust the back-projection is worthy of further 
investigation. Regarding the results of research on distance measurement with ultrasonic sensors [6, 7], 
an idea to apply ultrasonic sensors on this system 1s desriable. In general, research on distance measurement 
with ultrasonic sensors is used in flat material [7, 8] while human skin has various surface shapes. Therefore, 
the process of testing the back-projection method will be carried out using objects with various surfaces in 
this study. 


2. VEIN SPECTRUM ABSORBTION 

Veins are blood vessels used for intravenous therapy. Blood in these vessels contains Deoxy Hb, 
while blood in arteries contains Oxy Hb. Based on the results of research on the absorption of the light 
spectrum that shown in Figure l(a), it can be seen that veins and arteries have significant differences in 
absorption at wavelengths of around 680 nm [9]. On the other hand, there is a study developed by Travelakiz 
[10] to determine the absorption of the light spectrum in several paint samples shown in Figure 1(b). If 
observed at a wavelength of around 680 nm, it can be seen that Ultramarine Blue, Azurite, and Chromium 
Green have properties similar to Deoxy Hb. Whereas Vermilion, Cadmium Red, Chrome Yellow, and Zinc 
White have properties similar to Oxy Hb. 
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Figure |. (a) NIR spectrum absorption between Hb and HbO2 [9]; (b) NIR spectrum arbsorption in many 
samples paint [10] 


3. RESEARCH METHOD 

This study focused on development of real-time veins visualization with high mobility and 
flexibility. In order to achieve the high flexibility, back-projection concept that discussed in previous research 
[5] was necessary to implemented in this system. In this method, the distance is obtained by the ultrasonic 
sensor has an important role for determining the projection location. The distance is obtained by measuring 
time of flight of sound waves multiplied by sound wave velocity at the air [11]. However, this measurement 
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concept does not always get accurate results because the surface of the skin is not always flat. Therefore, 
several simulation objects with different surfaces as shown in 

Figure 2 are used to test the performance of the back-projection method. These objects were made 
by acrylic material to create the shape. Simulated veins created by blue lines according to Figure 1(b) where 
the blue color will absorb infrared light as same as deoxyhemoglobin. This blue line was printed on a sketch 
paper and placed on the surface of the shape. 

Figure 3(a) shows the system design in this study. The camera and ultrasonic distance sensor were 
used as input from this system. The frame produced by the camera was processed the microprocessor 
(Raspberry Pi 3B). The stages of image processing in this system consist of preprocessing, segmentation, 
and back-projection. The difference between veins and skin were strengthened in the preprocessing stage. 
The vein pattern was separated from the skin in the segmentation process before it projected in the back- 
projection stage. A mini projector was used as the output of this system to perform the back-projection 
method. 
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Figure 3. (a) System design; (b) Measurement setup to test the ultrasonic sensor 


3.1. Vein Image Acquisition 

The NoIR camera is a camera without an IR cut filter where the camera is able to see a near-infrared 
light spectrum (650-900) nm. When NIR light transmitted on the skin, it was scattered [12]. Some lights are 
reflected and some are forwarded inside due to the scaterization effect. Based on the results shown in 
Figure I(a), the light that can be passed into the veins (near infrared ray) will be absorbed. The absorption 
process will cause dark colors in veins when observed with a NoIR camera. Therefore, the use of NIR 
spectrum [13, 14] has important role to obtain the vein pattern. 


3.2. Preprocessing 

The difference between the veins and the skin produced by the camera does not look so clear to 
identify. Therefore, the difference between veins and skin needs to be strengthened. Note that the frame 
obtained by the camera has an RGB format. However, the application of visible light block filters causes the 
captured frame only contain gray levels. Since the next process requires a grayscale image, the RGB type 
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image is reduced to 1 layer (grayscale). Some low-pass filter [15, 16] was performed to handle the 
noisy image. 

Veins have a dark color because they absorb the NIR light given while the skin has a bright color. 
The contrast stretch process such as high-boost filter [17] or histogram equalization [18, 19] 1s widely used to 
reinforce the difference between veins and skin. Histogram equalization performs the calculation process 
obtained from all pixels in an image. Due to the distribution of vein patterns in an image is not uniform, 
the application of histogram equalization is less suitable because it can eliminate some information from vein 
images. Therefore, the use of adaptive histograms [19] or local histogram is more appropriate to use in 
this system. 


3.3. Segmentation 

The segmentation process aims to separate the vein pattern from the skin. The easiest segmentation 
method to do for a grayscale image is thresholding. However, general thresholding will affect each pixel in 
an image and allow lost information [20]. Therefore, the use of adaptive thresholding such as Niblack 
[21- 23] and Otsu [24] is expected to be able to overcome the lost information. Meanwhile, the threshold 
value for each block in this study is determined by the mean value of neighbors pixels minus center since the 
result give a good result and fast computation. 


3.4. Back-projection 

Vein image that already segmented will back-projected to the observation area. Based on the 
method used in this study shown in Error! Reference source not found.(a), a projection shift will be 
occurred if there is a change in distance between device and object. Therefore, a distance sensor is used as 
the main parameter to adjust the projection location. After the image processing is complete, the processed 
frame will be projected by determining the projection location (x,y) based on the distance between device 
and object. 
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Figure 4. (a) Back-projection method; (b) adjustment of processed frame to the original object location 


The projection location has been calibrated based on distance and a flat object which is expressed in 
(Error! Reference source not found.) and (Error! Reference source not found.). The processed frame 
was adjusted to original location area in the measured object as shown in Error! Reference source not 
found.(b). x and y are the location of processed frame in a full screen projector view with size of 1024*768 


pixels. 
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Since the skin does not have a fairly flat surface, it 1s necessary to observe the performance of 
back-projections on objects with different surfaceses as shown in 

Figure 2. All objects have the same length of 70 mm and were divided into several segments 1.e. 
S1, S82, ..., S7 as shown in Figure 4(a). The mechanism of measurement and placement of sensor against 


Veins projection performance based on ultrasonic distance sensor in various ... (I Putu Adi Surya Gunawan) 


1366 O ISSN: 2502-4752 


object is shown in Figure 4(b). Sensor position is perpendicular to the object base and fixed. The distance 
between sensor and base of the object is 30 cm. Meanwhile, the ultrasonic sensor can move verticaly in order 


to measure distance (d ) from S1 to S7. 
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Figure 4. Measurement scheme using ultrasonic sensor; (a) Segmented measurement using ultrasonic sensor; 
(b) Sensor and object position 


4. RESULTS AND ANALYSIS 

After several experiments, a discussion about result of vein image processing, back-projection on 
flat and multi surface is provided. Image processing algorithm was tested directly to the skin in order to 
obtain the vein pattern and performing back-projection as the last process. However, to observe the back- 
projection performance directly to the human skin is difficult to do since a static position of object is 
required. Therefore, a various surface objects as shown in Figure 2 was used to test the performance of back- 
projection method. 


4.1. Image Processing Result 

The results of the image processing process in each step are shown in Figure 5. Veins image 
obtained by the camera was not seen clearly where the difference between veins and skin was not strong 
enough. The information on veins image contains only dark and bright areas thus the grayscale process needs 
to be done. In emphasizing the difference between veins and the skin, adaptive histogram equalization 
provides good results where the veins become slightly clearer. 

In the segmentation stage, the vein pattern was able to be separated from the skin by using adaptive 
thresholding. However, there was also a segmented noise from the segmentation process. Morphology 
operation such as opening and closing was performed to remove the noise. Although the results were not 
significant, noise could be overcomed by the morphology process. Before being projected, a coloring process 
was applied to the results of morphology to avoid white lights. 

Previous research with dual camera configurations requires a PC with a high-speed processor to 
perform back-projection [4]. Meanwhile in this study, with the configuration in Error! Reference source 
not found..a, the whole process tooks between 65 ms. The computation time is relatively fast considering 
that the process was processed by a portable single board computer. The last process is to re-project the vein 
image onto the surface of the skin. The projection performance results will be discussed in the next section. 
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Figure 5. Image processing result (a) original image; (b) grayscale result; (c) adaptive histogram equalization 
result; (d) adaptive thresholding result; (e) morphology opening result; (f) morlphology closing result; 
(h) projected vein image result 

4.2. Result of back-projection on flat surface 

The test of back-projection at various distances with flat objects have excellent results. In this 
measurement, the position of the ultrasonic distance the sensor was kept perpendicular to the object even 
though the distance varies. From the results shown in Table 1, X locations have a maximum shift of 0.25 
mm whereas for y locations have a maximum shift of 0.48 mm. 


Table 1. Result of Measurement Using Flat Surface in Various Distance 


Distance (cm) 20 25 30 35 40 
x actual (pixel) 530 498 475 456 444 
y actual (pizel) -105 -46 -4 26 47 
X measured (pixel) 530 497 475 456 444 
y measured (pixel) -107 -46 -6 Zo 47 
x shift (mm) 0 0 0 0.25 0 
y shift (mm) 0.3 0 0.48 0.25 0 


4.3. Result of back-projection on various surface objects 


Figure 6(a) shows how the projection shift is obtained. A vein vessel is simulated by blue lines and 
the projected veins have a yellow color. A projection shift occurred when yellow lines overlap the blue lines. 
Since the vein diameter for IV access (cephalic veins) is range from 1.5 to 6 mm [25], a simulated vein size 
was set to 3.75 mm (median value). 

Figure 6(b) shows the projection shift that can still be tolerated. It can be seen that the unaccepted 
coverage area 1s covering less than 75% of vein vessel or maximum projection shift that can be tolerated is 
0.93 mm. Coverage area less than 75% will cause the needle insertion shift to the edge of the vein vessel or 
missed because the reference point is the center of projected veins. 
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Figure 6. (a) Projection shift measurement; (b) Projection coverage area 


Table 2 shows the results of projection shift on objects with multi-surface. From all objects, 
the most consistent projection shift occurred in each SI and S7 where these segments are close to the edge of 
the object. The projection shifts on each S1 and S7 may occur because these segments are not fully covered 
by the transducer. By ignoring the error in S1 and S7, some segments that have projection shift exceeding 0.9 
mm are in (C: 3,4,5), (D: 3), and (E: 6). This projection shifts occurred because the distance reading 
encountered a considerable error in these segments. 

The most projection shift occurred in S1 and S7 because the location of these segments is close to 
the edge of the object. While in the other segments, a large projection shift probably occurs due to large 
rotation angles to the sensor or steep curvature. Meanwhile, the other segments have projection shift lower 
than 0.9 mm. There are a few segments that almost have projection shift close to zero. Segments (A: 3,4,5), 
(D: 4,6), CE: 3), (F: 2,3,4,5,6), (G: 3,4,5), CH: 3,5), and (I: 2,4,5,6) have projection shift lower than 0.2 mm. 
It can be seen that the segments with a small projection shift have a less curved surface or the rotation angle 
between object and sensor is close to perpendicular. However, even though (H:3,5) have a curved surface, 
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the projection shift is lower than 0.2 mm. It 1s possibly occurred because the ultrasonic distance sensor also 
covers the neighbor of these segments and unfortunately made the distance reading have less error. 


Table 2. Result of Projection Shift Measurement in Millimeter 
Object 
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A test of the rotation angle between objects and sensors was conducted due to many projection shifts 
occur on the surface with a fairly high rotation angle. This test aims to know the characteristic of distance 
reading to the real distance in various rotation angle. Figure 3.b shows the measurement setup where there is 
a flat surface that can rotate from 10 to -10 degrees against the ultrasonic sensor. The distance between the 
object and the sensor was set at 30 cm. The measurement results with a 2-degree step are shown in Table 3. 
From these results, it can be seen that the sensor reading error was directly proportional to the rotation angle. 

In this study, objects B, F, and G have convex profiles, while objects I and J have concave profiles. 
The other objects that were not mentioned above have various curvatures. Therefore, objects B, F, G, I, and J 
are easier to analyze by arranging several forming lines. In its implementation, each segment has a forming 
line with different angles toward the ultrasonic sensor as shown in Figure 7. 


Table 3. Difference of Distance Reading to Real Distance in Flat Object Rotation 
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Figure 7. Multi angle forming line on each object 


Based on the results of the formed straight line, an approach can be made with the results of flat 
surface rotation as shown in Table 3. From the 5 objects shown in Figure 7, the highest rotation of forming 
line is +9 degrees. Objects B and J were formed with lines at the +9 degrees rotation in each segment. 
Object F was formed with lines at +6, +4, and O degrees rotation. Object G was formed with lines at +9, +6, 
and 0 degrees rotation. Object I was formed with lines with +5 and O degrees rotation. 

Table 4 shows the results of distance measurements of object B, F, G, I, and J. Objects B and I have 
close errors and can be classified as objects with the highest error because both have the same forming angle 
of 9 degrees in each segment. Objects F and G also have almost the same shape. A higher error has occured 
in Object G because there was a forming line of +9 degrees on S1, $2, S6, and S7. While object F in the same 
segment is formed by a line of +6 degrees. Object I has a smaller error than other objects because the forming 
line has an angle of 5 and O degrees. However, in its implementation only S2 and S6 have errors smaller than 
others, while S3, S4, and S5 have errors that are almost similar to object G which is formed by a line of -6, 0, 
and 6 degrees in the same segment. This result has the same characteristic as the results in Table 3 even 
though the value is not the same. So far it can be concluded that increasing the rotation angle of an object 
against the distance sensor will increase errors in distance reading. 


Table 4. Difference Between Real Distance and Measured Distance 
Difference between real distance and measured distance (cm) 


Object Sl S2 S3 S4 S5 S6 S7 
B 0.58 0.32 0.27 0.27 0.17 0.22 0.48 
F 0.44 0.13 0 0.07 0 0.13 0.44 
G 0.53 0.19 0.09 0.05 0.09 0.19 0.63 
I 0.5 0.1 0.19 0.1 0.14 0.1 0.5 
J 0.6 0.33 0.28 0.26 0.28 0.33 0.7 


From this analysis, so far it could be known that increasing the angle of a segment towards the 
sensor will also increase the error of distance reading. Since the location of the projection depends on the 
distance reading, increasing the angle of a segment will increase the shift in back-projection. The profile of 
human skin generally resembles objects F and G. Therefore, applying this system to the skin may actually 
have a projection error like those objects. By knowing the characteristics of F and G, the location of the 
sensor must be at the midpoint (S3, $4, S5) to minimize projection errors 1n the actual plant. 


5. CONCLUSION 

The performance evaluation of the proposed system in this study was successfully carried out. 
By utilizing a single board computer, the whole computations for 1 frame were successfully carried out with 
a total time about 65 ms (15 fps). The use of ultrasonic sensor in the projection method has good results when 
measuring flat surface object with maximum projection shift of 0.48 mm. The test results on several samples 
with various surfaces have good results where the projection shifts below 0.9 mm reach 89% of 140 
measurement points. Some large projection shift occurs because the distance sensor reading occurrences an 
error caused by the surface of the object that has a steep curvature or a large rotation angle to the sensor. 
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